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We focus on the study of dynamics of two kinds of random walk: generic 
random walk (GRW) and maximal entropy random walk (MERW) on two 
model networks: Cayley trees and ladder graphs. The stationary probabil- 
ity distribution for MERW is given by the squared components of the eigen- 
vector associated with the largest eigenvalue Ao of the adjacency matrix of 
a graph, while the dynamics of the probability distribution approaching to 
the stationary state depends on the second largest eigenvalue Ai. 

Firstly, we give analytic solutions for Cayley trees with arbitrary branch- 
ing number, root degree, and number of generations. We determine three 
regimes of a tree structure that result in different statics and dynamics of 
MERW, which are due to strongly, critically, and weakly branched roots. 
We show how the relaxation times, generically shorter for MERW than for 
GRW, scale with the graph size. 

Secondly, we give numerical results for ladder graphs with symmetric 
defects. MERW shows a clear exponential growth of the relaxation time 
with the size of defective regions, which indicates trapping of a particle 
within highly entropic intact region and its escaping that resembles quan- 
tum tunneling through a potential barrier. GRW shows standard diffusive 
dependence irrespective of the defects. 

PACS numbers: 05.40.Fb, 02.50. Ga, 89.75.Hc, 89.70.Cf; 

1. Introduction 

After Einstein [1J and Smoluchowski [2] gave explanations of Brownian 
motion and originated the theory of diffusive processes, there has been an 
unceasing research on models of random walk (RW) , which may be regarded 
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as time or space discretization of these processes. Thousands of papers and 
textbooks in statistical physics, particle physics, engineering, economics, 
biophysics, etc., have been and still are published. 

From mathematical perspective, RW is a Markov chain describing the 
random consecutive steps of a particle. As an example, the well-known Polya 
random walk on a lattice [3] at each time performs equiprobable steps to 
any of the neighboring nodes. This process, generalized to any graph, is 
known as the ordinary or generic random walk (GRW). 

RW can also maximize the entropy of paths, and hence we call it the 
maximal entropy random walk (MERW); lately, this type has been studied 
in [HE]. This principle of entropy maximization, which is a global one alike 
the least action principle, earlier brought about the biological concept of 
evolutionary entropy [6l [7] . It also served as an optimal sampling algorithm 
in the problem of importance sampling [8]. MERW has also begun to be 
used in the study of complex networks [9J [TUl [HI E2J [13] . 

The defining feature of MERW makes the paths of given length and 
end-points equiprobable. This leads to an unprecedented feature that the 
stationary probability on diluted lattices localizes in the biggest spherical 
region [H [5] . An interactive online demonstration [13] illustrates this fea- 
ture. In this paper, we focus on how the dynamics of GRW and MERW 
differs. More precisely, we show analytic expressions for stationary prob- 
ability distributions and relaxation times of GRW and MERW on Cayley 
trees; we also give numerical results for ladder graphs, showing that the 
relaxation time for MERW grows exponentially with the size of defective 
regions as opposed to diffusion behavior for GRW. 

In this paper, in Sec. 2 we provide definitions and notes on the two 
types of random walk. In Sec. 3, we give several analytical results concern- 
ing Cayley trees (involving eigenproblem solution for the adjacency matrix, 
discussion of stationary state and relaxation). Lastly, in Sec. 4, we show 
numerical results concerning relaxation process on a class of ladder graphs. 

2. General considerations 

Let us consider a discrete time random walk defined by a constant 
stochastic matrix P, on a finite connected undirected graph. The prob- 
ability that a random walker which can be found on a node i at time t hops 
to a node j at time t + 1 is encoded by the element Pij > of this matrix. 
Another condition fulfilled by these matrix element is ^ ■ Pij = 1 for all i. 
If we denote by A the adjacency matrix of the graph (Aij = 1 if i and j 
are neighbors, and = otherwise), we can formulate an additional con- 
dition: < A^, which means that particles are allowed to jump between 
neighboring nodes only. The stochastic matrix corresponding to the generic 
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random walk (GRW) is given by: 

r« = ^ , (i) 

where ki = J2j Aij ^ s the node degree, and the probability of selecting one 
of ki neighbors of the node i is uniform. This means that the entropy of 
neighbor selection is maximized and shows that this is the standard Einstein- 
Smoluchowski-Polya random walk. Lastly, the stationary state of GRW is 
given by 7Tj = h/Yj k j- 

On the other hand, maximal entropy random walk (MERW) maximizes 
the entropy of choosing a trajectory of given length and end-points. This 
principle leads to 

13 ~ A Vo* ' [ ) 

where Ao is the largest eigenvalue of the adjacency matrix A and is the 
z-th component of the corresponding eigenvector ipQ. From the Frobenius- 
Perron theorem and from the fact that the adjacency matrix A is irreducible 
it follows that all elements of ip$ are strictly positive. Shannon-Parry mea- 
sure [15] then describes the stationary state of P: 

■Ki = ifa . (3) 

Intriguingly, this equation forms a connection between MERW and quantum 
mechanics, as one may interpret i/jq as the wave function of the ground state 
of the operator —A and consequently ip^ becomes the probability of finding 
a particle in this state [HE]. The two random walks, ([1]) and ([2j), in general 
exhibit altogether different behaviors except for the case of /c-regular graphs, 
where they coincide. 



3. Cayley tree 

We define a Cayley tree with a branching number k, which is the number 
of edges leading from a given node to the next generation of nodes, and the 
number of generations G. The root of the tree is assumed to have a degree 
r and it belongs to the zeroth generation (see Fig. [1]). The number of nodes 
in the zeroth generation is therefore no = 1, in the first n\ = r nodes, in 
the second ri2 = rk, in the third one = rk 2 , etc. The tree has n nodes in 
total: n = ^ =0 n g = 1 + r(k G - l)/(k - 1). 
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Fig. 1. Cayley tree with root degree r = 3, branching number k = 2, and G = 3 
generations. 

3.1. Eigenvalues of the adjacency matrix 

This section is devoted to calculation of eigenvalues of the adjacency 
matrix of Cayley tree, which can be determined by solving the equation 

det(A - Al) = (4) 

The determinant can be calculated with the use of a sequence of elementary 
transformations that leave it invariant, e.g., additions of multiple of a row 
or column to another row or column. Thus, the determinant can be reduced 
to a triangular form with zeros above the diagonal, as first presented in [16j . 
Details of this procedure can be found in |17j . The triangular form of the 
determinant allows to rewrite (j3|) as a product of the diagonal coefficients 

K(A)P = 0, (5) 

3=0 

where = 1 and mc- g = n g — for g = 1, 2, . . . , G, and A g (X) are 

polynomials w.r.t. A given by the recursive equations 

A (X) = -A, 

Ag(X) = -XA g ^(X)-kA g _ 2 (X), for g < G, (6) 
A G (X) = -XA G ^(X)-rA G ^ 2 (X). 

Notice that for g = G the coefficient k is replaced by r, which is a conse- 
quence of the tree structure allowing arbitrary root degree. To complete the 
set of equations we have to take initial condition A-i = 1. The real roots 
of equation ([5]) counted with the degeneracy m g give the total number of 
Ylgid + l) m g = J2 g n g = n i which means all n eigenvalues of the adjacency 
matrix are retrieved. 

The recurrence ([6]) can be solved: 

Ag = k ( 9+ l)/2 M(9 + 2)9] j fQrg<G (?) 
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where cos 9 = -\/(2y/k) and 9 is an auxiliary parameter. To obtain the 
polynomial Aq one needs combine the last equation in © Aq = —XAq-i — 
kAc-2 with the solutions for Aq-i and Aq-2 0, which yields 



A 



G 



/-(G-l)/2 



k sm[(G + 2)9] + (k - r) sin(G#) 



sm( 



(8) 



Now, instead of ([5]) we can consider equations A g = and Aq = to 
find the solutions for 9 and then determine the eigenvalues of the adjacency 
matrix using the formula A = — 2^/kcos9. The first equation, leading to 



sm[(g + 2)9] = 0, 



has g + 1 solutions 



V 9J 



2\fk 



cos 



g + 2 



for i = 1,... ,g + 1. 



(9) 



(10) 



As follows from ([5]), each eigenvalue A 9i j in this series has multiplicity m g . 
The equation ^4^ = produces 

k sin[(G + 2)9] + (k - r) sin(G0) = 0, 

for which analytical solutions exist in the case r = k 



Agj = 2\/fc cos 
and in the case r = 2k 

Xg j = 2\/A;cos 



G + 2 



*U - 1/2) 
G + l 



for j = 1,...,G + 1, 



for j = l,...,G + l . 



(12) 



(13) 



Other choices of r involve numerical solving of (jlip . 

It can be shown that the largest eigenvalue of the adjacency matrix is 
Ao = Ag,i- It belongs to one of three classes of solutions depending on r, 
which takes values r € (0, 2k — 2k/G) in the first class, r G (2k — 2k/G, 2k + 
2k/G) in the second, and r G (2k + 2k/G, +oo) in the third. 

For large G (i.e., G 3> 2k) the second interval becomes just a single 
integer value r = 2k. The first class allows values r < 2k for which an 
approximate solution exists 



A n = 2y/k 



7T 



COS 



G + <5' 



(14) 



where 5 ~ 2k /(2k — r), or the exact solution for r = k (|12p . In the third 
class, r > 2k, there are no real solutions of (|11|) for G (0,7r/(G + 1)) and 



G 
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Ao corresponds to a purely imaginary 9. The trigonometric equation (jlip is 
thus replaced by a hyperbolic one. In the limit of large G the approximate 
solution is 

2y/k 



Ao = 7==r , (15) 



x 



where 



l + z 



G+l 



2k . . 

and z = 1 . (16) 



3.2. The eigenvalues of the GRW transition matrix 

The stochastic matrix of generic random walk ([I]) can be subjected to 
the same procedure as explained in Sec. 13.11 Transforming its determinant 
to the triangular form generates analogous recursion as in ([6]): 



A = -A, 

k 

{k + 1 



A g = -XA g - 1 - {u | n2 Va. for 5 = 2,...,G-1, (17) 



A G = -A^ G _i - -^yA G _ 2 . 

We take A_i = + 1 as an initial condition that agrees with the rest of 
equations and proceed as before solving this recurrence to obtain eigenvalues 
from the equations A g = and Aq = 0. From Aq = one gets 

2/ccos(260 = 1 + k 2 and sin(G0) = 0, (18) 

whose solutions lead to, respectively, 



A = 1, and X GJ = 2y — ^ cos (j^J , for j = 1, . . . , G, (19) 
and from A g = 

sm[(g + 2)e] = ksm(g9). (20) 

The last equation has identical form as (jlip except for different coefficients. 
The class of solutions of (fTTj) with r £ (2/c + 2k/ G, +oo) corresponds to 
value fc > 1 in the above equation. Hence, the value of 6 that leads to the 
largest eigenvalue in a given series is imaginary. Once again, the trigono- 
metric equations (|20|) change into hyperbolic ones. Upon replacements 
k = f±l, z = §=±, we end up with (H5J rewritten as a; = |^ [l - 2k~^+ 1 )] . 
In large G limit, the second largest eigenvalue is thus approximated by 
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and clearly the second largest eigenvalue Ai approaches Ao = 1 exponentially 
in G. 

3.3. Stationary states of GRW and MERW on Cayley trees 

A random walk on a graph has a stationary probability distribution if 
the graph is not bipartite. If a graph is bipartite, one can define a semi- 
stationary state: it involves either averaging probability distributions over 
two consecutive time steps t and t + 1 (because the distributions for even 
and odd times are independent) or averaging the distribution over initial 
conditions. 

GRW leads to the stationary occupation probabilities 

k- 

n = ^rr , for i = 1, . . . ,n, (22) 

which comprise a flat distribution for nodes of degree ki = k + 1 and the 
exception of the root having r neighbors and leaves neighboring with just 
one node. As nodes in each generation have equal stationary probabilities 
we can sum over them U g = n g Tii oc k 9 ~ l , which produces the exponential 
factor. 

The stationary probabilities of MERW are equal to the squared com- 
ponents of ipQ. All elements ipQi of this vector have the same values for i 
belonging to a given generation g. This simplifies the description of the sta- 
tionary state so that we may write ip g for all nodes in the generation g (we 
omit the first index, which numbers the corresponding eigenvalue). Exact 
solution for ijj g can be obtained by solving a recurrence equation analogous 
to ®: 

7Ti = Voi oc k G ~ 9 sin[(G - g + 1)9] 2 , for g = 0, . . . , G and i £ g, (23) 

where the normalization constant has been omitted. After summing over 
whole generation i € g, the probabilities become 

U g = ngiri oc k G ~ 1 sm[(G-g + 1)6} 2 , for g = 1, . . . , G and i € g, (24) 

where the case g = with its no = 1 needs a separate treatment. 

This result depends on the choice of r, k through 9 and A. For r < 2k, 
parameter 6 ~ and the limiting distribution is a sine square; for r = 2k, 

6 = qt^ and the distribution is a cosine square; for r > 2k, 9 = i arctanh x 
[where x is defined in f)16[) . while i is the imaginary unit], which yields 
a hyperbolic sine. These limiting results as well as finite-size effects are 
showed in an online interactive demonstration |18j . 
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3. 4- Relaxation times 

A stochastic matrix needs not be symmetric, thus its right and left 
eigenvectors may differ: P^q = A Q $ tt , $ a P = A a & a . Hence, there exists a 
spectral decomposition of P 

Pij = J2A a y ai <f> aj , (25) 

a 

where for MERW one can make replacements: A a = A Q /Ao, = V'mAAoi) 
= V'mV'Oi- The spectral decomposition of the adjacency matrix of a 
given graph thus contains information about all properties of MERW. 

From the knowledge of the initial probability distribution 7?(0) and the 
transition matrix P the distribution can be determined at any time t 

vf(t) = 7f(0)P*, (26) 

where the elements 7Tj(i), i = l,...,n of 7r(t) denote the probability of 
finding a particle performing a random walk at a node i at time t. 

The last equation can be reformulated utilizing the spectral decomposi- 
tion of the stochastic matrix (|25p 

7?(i) = J>X$a- (27) 

a 

where c a denotes a spectral coefficient: c a = 7?(0) • ^ a = Y^i ^i(^)^ai- 

Generally, all eigenvalues A a of P are located inside or on the unit circle 
in the complex plane |A Q | < 1 and in the limit of infinite t on the right- 
hand side of (I27p only |A a | = 1 survive, while all the other terms vanish 
exponentially. 

For both GRW and MERW on a tree only two eigenvalues on the unit 
circle are left Ao = 1 and A n = — 1 due to bipartiteness of the graph. 
For t — > oo the relaxation to the stationary state is generically governed 
by the second largest eigenvalue Ai and its negative counterpart A n _i = 
— Ai. The corresponding term in the spectral decomposition (j27|) decreases 
exponentially as exp(— t/ri), where T\ = [— ln(Ai)] -1 = [ln(Ao/Ai)]~ . 

Thus, T\ is what we call the generic relaxation time, which is the largest 
one. We note, however, that there are symmetries that lead also to other 
relaxation times. As the eigenvalues of the adjacency matrix depend on the 
tree parameters, also the relaxation times for MERW fall into several classes. 
The relaxation times for large G are given in the Table [TJ It is noteworthy 
that whereas the probability distribution for GRW relaxes linearly with the 
system size t\ ~ n ~ k G , for MERW it is as fast as a logarithm of the 
system size n ~ Inn. Derivations and further details expanding the note 
on symmetries can be found in [17] . An online interactive demonstration 
|19| can also facilitate understanding of these results. 
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Regime 


Ao 


Ai 


Tl 


Strongly branched: 








r > 2k, k > 1 


Eq.flSJ 


Ag-i,i 


, C 2 7T 2 1 , 

2 G 2 + ' ' ' 


Critically branched: 






r = 2k, k > 1 


Ag,i 


Ag-i,i 


8G 2 , 16G , 


Weakly branched: 








1 < r < 2k, k > 1 


Ag,i 


Ag-i,i 


2fc-r/-t3 , 3(4fc-r)^2 


Planted tree: 






r = 1 


« Eq.dUD 


Ag-2,1 


||4 G 3 + 3 G 2 + 


Linear chain: 








k = 1, r = 1 


Ag,i 


Ag,2 


2G 2 , 8G , 


GRW: r > 1, k > 1 


1 




8fe r^G 
(fc-l) 2K 



Table 1. Relaxation times T\ for large G. All rows except for the last one refer to 
MERW. The symbols X g j correspond to one of the equations (|10I) .([T2 |) . or (|T3l) . 
whichever is appropriate for the choice of parameters k,r. In the first row: c 
: 1 1 i — r While the number of vertices n ~ fc G , the probability distribution 

relaxes a logarithm of the system size ti ~ Inn. 



4. Ladder graph 

In this section, we discuss a particular class of ladder graphs (exemplary 
ladder graph can be seen in Figj2]). A ladder graph consists of two chains 
of integer length n/2 which are connected by rungs, i.e. node % of one chain 
is connected to node %' of the second one, then i + 1 to i' + 1 and so forth. 
We also impose periodic boundary conditions producing a ring, where node 
i + n/2 is connected to node i + 1, and node i' + n/2 to node i' + 1. This 
structure is symmetric with respect to reflection i — > i' , and so the graph is 
a quasi one-dimensional system. It is a 3-regular graph, although we remove 
some rungs from the ladder to introduce defects, so that MERW and GRW 
are not equivalent on this graph anymore. 

For the adjacency matrix of the graph A, its largest eigenvalue Ao and 
the eigenvector ipQ associated with it, the stationary solution for MERW is 
given as the ground state of the tight-binding equation 

(h^o) = (" A ^o) a + V a ^o, a = (28) 

where the Hamiltonian is defined as H a i, = k max S a b — A a b, with the Kro- 
necker delta 8 a b, maximum degree of the graph A; max , and V a = A; max — 
k a , Eq = fc max — Aq. For a ladder graph with defects this equation yields 



2^0,a - lf>0,a-l ~ V>0,a+1 + V a 1p ,a = E ^0,a 



(29) 
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Fig. 2. (Colors online) Stationary probability for a ladder graph with periodic 
boundary conditions: the probability localizes in the intact regions. Red nodes 
represent initial condition that would be chosen for this graph. 



where Eq = 3 — Ao and V a = or 1 (rung present or absent). Stationary 
states of a number of ladder graphs (with one, two, or a number of random 
defects) were discussed in Section 6 of [5]. 

Additionally, we impose a symmetry on those defects: there can only be 
two equal regions intact and two equal regions with rungs removed (gaps). 
We take the initial probability 1 at the center of one of the intact regions 
(this may be 2 or 4 nodes, depending on whether the length of the region is 
odd or even, see FigJ2|). The systems under study have n = 48 — 512 total 
number of nodes and the number of deleted rungs separating two regions 
(the gap size) varies between g = 1 — 10. 

We measure the probability P(t) summed over one whole region (as the 
regions are equal in size, P^ = 1/2 is its stationary value). It might be 
understood as a macroscopic measure of the process taking place in this 
region. As expected, the probability flows from one the initial intact region 
to the other one until equilibrium (P(t) = 1/2 in both regions) is attained. 

We fit the numerical results to exponential dependence on time t: P(t) ~ 
exp[— a(t — &)], where a and b are fitting parameters from which we extract 
the relaxation time r, which is the characteristic time scale of an exponential 
approach to the stationary state. The results for the behavior of relaxations 
times for GRW and MERW are given in Table It turns out that for 
MERW there is a clear dependence of the relaxation on the gap size for a 
given lattice size (example in Fig. [3|a) for n = 96): a(g) = exp(— c • g — d), 
b(g) = exp(c ■ g + d), where c, d are constants with respect to the gap size 
g. After extracting this dependence, only the dependence on the system 
size remains in the function c = c(n), which is very well fitted with a 
power law [Fig. Efb)]: c(n) = Cqo — fn~ l l u (best-fit value parameters are 
Coo = 0.9643±0.0078, / = 58±42, v = 0.773±0.098). Thus, the macroscopic 
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GRW 
MERW 



r(n, g) = c • n , d = const. 
r(n, g) = exp [c(n) • p] 



Table 2. 

Relaxation times r as functions of the system size n and gap size g, where 
c(n) = Coo — $rT x l v and d, Cqo, /, f are fitted constant. 



1 x10 6 




200 300 400 

Lattice size n 



Fig. 3. Maximal Entropy Random Walk: (a) Logarithmic plot shows an exponential 
dependence of the relaxation time on the gap size (an exemplary system size, 
n = 96) reminding of quantum tunneling, (b) the dependence of the relaxation 
time on the system size, c(n) — — jn~ x l v [see (|30[)], Continuous lines are the 
best hts of an exponential function and power law, respectively. 



probability depends on time, system size, and gap size: 

\P(t;g,n) - P^l oc exp {-exp [-c(n) ■ g] ■ t} . (30) 

For GRW, both fitted parameters a and b have shown no dependence 
from the gap size g, although they do depend on the system size: a(n) = 
cn~ d , b(n) = c'n d , where d',d ~ 2 (see FigU]). This produces the familiar 
behavior r ~ n 2 which is expected for a one-dimensional random walk. 

5. Conclusions 

In this paper, we have discussed the dynamics of generic random walk 
and maximal entropy random walk on two classes of graphs. For Cayley 
trees, we have shown the analytic form of generic relaxation times governing 
how fast probability distributions of those random walks approach their 
stationary states. MERW has proven to be generically faster (logarithmic 
with respect to the system size) than GRW (linear w.r. to the system size). 
However, on defective ladder graphs the relaxation of probability seems 
to show opposite behavior: while GRW relaxes diffusively, the relaxation 
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(b) 

» » n i 



50 100 200 500 2 4 6 8 10 

Lattice size n Gap size g 

Fig. 4. Generic Random Walk: (a) log-log plot shows power law dependence of 
relaxation time, expected for a diffusion process (continuous line is the best fit; 
gap size g — 1), (b) the best-fit exponents d of the power law show independence 
from the gap size. The errors result from finite-size effects. 

times for MERW are much longer, growing exponentially with the size of 
the defective region. 

These results indicate that MERW might exhibit comparatively fast 
relaxation within intact or homogeneous regions (like a Cayley tree) but 
inhibits the relaxation process between regions separated by defects, bottle- 
necks or bridges. While qualities of MERW's stationary states have already 
been utilized to improve centrality measures in complex networks [10], its 
dynamic properties and a close relation between eigenvalues of the adja- 
cency matrix and the statistics of paths may be of use in community search 
algorithms on complex networks (a number of algorithms based on random 
walks, path enumeration and spectral properties of the adjacency matrix are 
reviewed in |2Uj). As a more speculative idea, it is also worth remembering 
that MERW keeps all paths of a given length between any two endpoints 
equiprobable, which makes it capable of hiding the route information trav- 
els, e.g. over the Internet. 
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